Clustering Curves in the Presence of Heteroscedastic Errors

The clustering technique introduced in this paper is a means for discovering underlying patterns among a large number of curves. One novel characteristic compared to the current clustering methods is that we allow for heteroscedastic errors. Both the mean and the variance functions of each curve are assumed unknown and varying over time. The clustering method consists of a series of steps: transformation using an orthonormal basis of functions in L2, dimension reduction through coefficient estimation in the transform domain, and clustering in the transform space. We show that in the transform space, the coefficient estimation procedure introduced in this paper is asymptotically optimal in the Pinsker’s minimax sense over Sobolev ellipsoids. We illustrate our technique by clustering a large number of curves both within a synthetic example and within a specific application. In this application, we analyze the research and development expenditure over time of a subset of companies in the Compustat Global database. We show that our method compares favorably to two alternative approaches.